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DETAILED ACTION 

Response to Arguments 

On page 8 of the Applicant's Response, Applicant argued that the Examiner 
failed to address any of the arguments related to the combinability in previous 
Response filed 03/25/09. With regards to the Response filed on 03/25/09, on page 14, 
Applicants argue that the combination of Yen and Brodsky is improper because Brodsky 
requires significant user interaction and thus to assert a user would be enabled to 
quickly and efficiently retrieve supplemental information would be illogical. 

Yen teaches a system for receiving incoming information from multiple sources, 
both interactive and passive, and for presenting that information to a user. The tuned 
broadcast signal comprises information in addition to the broadcast television shows 
such as annotation for those shows. Thus, the system provides the user with additional 
information related while the show is being presented to providing the user with an 
interactive viewing experience (Col. 3 lines 16-34, Col. 4 line 65-Col. 5 line 7). Yen 
further discloses the system uses a background element 121 and foreground element 
1 22. The background element and the foreground element work in together for 
presenting relevant information to the user. When the background element receives 
relevant information, it transmits the information item to the foreground element, which 
can immediately begin the displaying the information item without user interaction (Col. 
1 1 lines 42-57). However, Yen failed to teach that the received supplement data relates 
to one or more key phrases in the closed-captioning script. 



Application/Control Number: 10/647,549 Page 3 

Art Unit: 2427 

Brodsky was relied upon to teach this limitation since Brodsky teaches that 
keywords are extracted from the closed-captioning to develop a dictionary of keywords 
that can be used to request additional information (Col. 1 lines 50-62, Col. 2 lines 20-41, 
Col. 5 lines 36-63, Col. 6 lines 12-42). Thus, it would have been obvious to one of 
ordinary skill in the art to apply the technique of receiving supplemental data that relates 
to one or more key phrases of a closed captioning script as taught by Brodsky, to 
improve the information retrieval system of Yen for the predictable result of enabling the 
user to retrieve supplemental information pertaining to a word or group of words just 
heard (Col. 3 lines 52-60). The argument regarding to the motivation to combine 
references is moot in view of a new motivation provided in office action mailed 06/09/09. 

On page 9 of the Applicant's Response, Applicants argue that Brodsky does not 
teach that the supplemental data relates to one or more key phrases in the closed 
captioning script of a program, but merely teaches that the closed captioning script is 
used to generate a dictionary. 

The Examiner agrees that closed captioning script is used to generate a 
dictionary that stores list of word and phrases from the program that is currently in 
progress. Users may then request additional information about words or phrases heard 
within the program, such as when a newscaster utters "map of France", the user can 
request additional information and the recognition system will match the word or phrase 
to words/phrases within the dictionary and then retrieve the additional information from 
an outside source. The retrieved information is presented as an overlay on the TV 
broadcast via picture-in-picture, split screen, or by other means (Col. 6 lines 12-42). 
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Thus, since the words in the closed-captioning script are used to build the dictionary 
and then are directly used to retrieve additional information Brodsky meets the limitation 
that "the supplemental data relates to one or more key phrases in the closed captioning 
script of a program." 

On page 9 of the Applicant's Response, Applicants further argue that Brodsky 
when coupled with Yen and Hidary would serve to frustrate the methods of Yen and in 
particular those of Hidary. Since, Hidary discloses a method that synchronizes web 
pages with video content and Brodsky's discloses a dictionary of "recently heard words", 
thus the retrieved supplemental content could not be synchronized with the video 
because the portion with which it would synchronize has already past. 

The Examiner respectfully disagrees with Applicant's arguments because while 
Brodsky discloses that the dictionary is created using keywords recently heard in the 
closed-captioning script which does not necessarily suggest that the scene has already 
past, but merely that the spoken dialog has already occurred. Thus, when a user has 
heard "France" the user could immediately retrieve additional information about France 
during the newscast scene about France. Furthermore, Brodsky teaches a means to 
anticipate requests for a match by relaying "words" already in the buffer by 
preprocessing searches for those "words", so that in the event a user makes a request 
for additional information the system can satisfy the request more quickly (Col. 5 lines 1- 
1 0). Therefore, with the Brodsky system users can retrieve information related to the 
current scene. 
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On pages 12-14 Applicant argues that Chen fails to teach or suggest that "each 
of the identified key phrase has the same number of words." 

The Examiner respectfully disagrees with Applicant's argument because Chen 
teaches that processor 1 1 examines the tokenized document and generates multiword 
phrases. Processor 11 extracts from each sentence non-overlapping phrases of two or 
more words (Col. 3 lines 3-25). Thus, each identified phrase can have the same 
number of words, in this case each identified key phrase can have exactly two words. 
Furthermore, the Examiner fails to see how the teaching of Chen's "two or more words" 
is wholly indefinite, as it is clear that the non-overlapping phrases selected by processor 
1 1 can be phrases with only two words. Therefore, Chen meets the limitation that "each 
of the identified key phrase has the same number of words." 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-3, 5 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yen et al. (US Pat. 6,668,278), in view of Brodsky (US Pat. 5,809,471), herein 
referenced as Yen and Brodsky, respectively. 

Regarding claim 1, Yen discloses "a viewer computing unit (information receiver 
110) having a processor, a program enhancement listener implemented as computer 
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executable instructions stored on a computer-readable medium and executable on the 
processor (Col. 4 lines 26-35, Fig. 1 ) to direct the viewer computing unit to: receive 
supplemental data sent to a multicast address (Col. 4 line 47-Col. 5 lines 7, i.e. the 
television signal is broadcasted, which can be considered multicast if the multicast is 
sent to all users), wherein the received supplemental data relates to ... a closed 
captioning script" (Col. 3 lines 16-34, Col. 4 line 65-Col. 5 line 7, Col. 5 lines 40-50, Col. 
8 lines 34-49, Col. 11 lines 20-31 , i.e. Yen teaches of a system that receives 
supplemental content, such as web links, provided in the closed-captioning script). 

"initiating an enhancement action based upon the supplemental data to enhance 
the program as the video program is being played and to synchronize the supplemental 
data with scenes in the video, wherein the enhancement action is performed without 
user interaction" (Col. 7 lines 25-40, Col. 9 lines 33-50, Col. 1 1 lines 20-31 , lines 42-57, 
i.e. audiovisual material is synchronized with the program. For instance, during the 
commercial for a Ford truck a link to Ford's website is displayed). 

Yen fails to explicitly disclose that "the received supplemental data relates to one 
or more key phrases of a closed captioning script of a video program". 

Brodsky discloses that "the received supplemental data relates to one or more 
key phrases of a closed captioning script of a video program" (Col. 1 lines 50-62, Col. 2 
lines 20-41, Col. 5 lines 36-63, Col. 6 lines 12-42, i.e. Brodsky teaches that keywords 
are extracted from the closed-captioning to develop a dictionary of keywords that can be 
used to request additional information). Thus, it would have been obvious to one of 
ordinary skill in the art to apply the technique of receiving supplemental data that relates 
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to one or more key phrases of a closed captioning script as taught by Brodsky, to 
improve the information retrieval system of Yen for the predictable result of enabling the 
user to retrieve supplemental information pertaining to a word or group of words just 
heard (Col. 3 lines 52-60). 

Regarding claim 2, Yen discloses that "the enhancement action comprises 
activation of a hyperlink" (Col. 3 lines 16-34, Col. 5 lines 40-50, Col. 11 lines 20-32). 

Regarding claim 3, Yen discloses that "the enhancement action comprises 
launching executable code" (Col. 3 lines 16-34, Col. 5 lines 40-50, Col. 11 lines 20-32, 
i.e. Yen teaches of activating a hyperlink to a website, which reads on "launching 
executable code"). 

Regarding claim 5, Yen discloses that "computer-executable instructions to 
direct the viewer computing unit to display the supplemental data concurrently with the 
primary content" (Col. 9 lines 27-50, Col. 11 lines 20-32, lines 42-57, i.e. the 
background and foreground elements determine if information items can or should be 
displayed simultaneously). 

Claims 6-7 are rejected under 35 U.S.C. 103(a) as being unpatentable over Yen 
in view of Brodsky, and in further view of Hidary et al. (US Pat. 5,774,644), herein 
referenced as Hidary. 

Regarding claim 6, Yen fails to explicitly "[presenting] the video program within a 
hypermedia document; and controlling placement of the video program within the 
hypermedia document using the supplemental data". 
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Hidary discloses "[presenting] the video program within a hypermedia document; 
and controlling placement of the video program within the hypermedia document using 
the supplemental data" (Col. 7 lines 10-29, i.e. teaches of a JAVA enabled browser that 
allows a computer to retrieve web pages from a video program. The retrieved web 
pages are then synchronized with the video content). Thus, it would have been obvious 
to one of ordinary skill in the art to apply the technique of presenting a video program 
within a hypermedia document and controlling placement of the video program within 
the hypermedia document using the supplemental data as taught by Hidary, to improve 
the information retrieval system of Yen for the predictable result of providing the user 
with a more interactive experience while watching television by providing them with 
additional information corresponding to the program. 

Regarding claim 7, Yen discloses "a viewer computing unit having a processor 
and a display" (Col. 4 lines 26-35, Fig. 1). 

Yen fails to explicitly disclose "a hypermedia document stored on computer- 
readable medium and executable on the processor for graphical rendering on the 
display, the hypermedia document containing the program enhancement listener". 

Hidary discloses "a hypermedia document stored on computer-readable medium 
and executable on the processor for graphical rendering on the display, the hypermedia 
document containing the program enhancement listener" (Col. 7 lines 10-29, i.e. 
teaches of a JAVA enabled browser that allows a computer to retrieve web pages from 
a video program. The retrieved web pages are then synchronized with the video 
content). Thus, it would have been obvious to one of ordinary skill in the art to apply the 
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technique of storing a hypermedia document on computer-readable medium and 
executable on the processor for graphical rendering on the display, the hypermedia 
document containing the program enhancement listener as taught by Hidary, to improve 
the information retrieval system of Yen for the predictable result of providing the user 
with a more interactive experience while watching television by providing them with 
additional information corresponding to the program. 

Claims 8, 10-15 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hidary et al. (US Pat. 5,774,644), in view of Brodsky (US Pat. 5,809,471), and in further 
view of Yen et al. (US Pat. 6,668,278). 

Regarding claim 8, Hidary discloses "determining an insertion point there in for 
the supplemental content; sending the supplemental data to a device having a multicast 
address; receiving, by the device having the multicast address, supplemental data; 
initiating, at the determined insertion point, an enhancement action based upon the 
received supplemental data to enhance a video program as the video program is being 
played" (Col. 7 lines 10-40, Col. 7 line 65-Col. 8 line 4, i.e. Hidary teaches of URL's that 
are transmitting via a multicasting connection. The web pages referenced by the URL's 
are time stamped to be displayed when predetermined related video content is 
displayed). 

Hidary fails to explicitly disclose "automatically and directly associating 
supplemental data with at least one key phrase in a closed captioning script by parsing 
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the closed captioning script and... wherein the enhancement action is initiated without 
user interaction". 

Brodsky discloses "automatically and directly associating supplemental data with 
at least one key phrase of a closed captioning script by parsing the closed captioning 
script" (Col. 1 lines 50-62, Col. 2 lines 20-41, Col. 4 lines 4-17, Col. 5 lines 36-63, Col. 6 
lines 12-42, i.e. Brodsky teaches that keywords are extracted from the closed- 
captioning to develop a dictionary of keywords that can be used to request additional 
information. The system automatically creates the dynamically changing dictionary that 
associates supplemental data with the key phrases). Thus, it would have been obvious 
to one of ordinary skill in the art to apply the technique of receiving supplemental data 
that relates to one or more key phrases of a closed captioning script as taught by 
Brodsky, to improve the enhanced video programming system of Hidary for the 
predictable result of enabling the user to retrieve supplemental information pertaining to 
a word or group of words just heard (Col. 3 lines 52-60). 

The combination of Hidary and Brodsky still fail to explicitly disclose that "the 
enhancement action is initiated without user interaction". 

Yen discloses that "the enhancement action is initiated without user interaction" 
(Col. 11 lines 42-57, i.e. Yen teaches that the foreground element can immediately 
begin displaying the information item). Thus, it would have been obvious to one of 
ordinary skill in the art to apply the technique of initiating an enhancement action without 
user interaction as taught by Yen, to improve the enhanced video programming system 
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of Hidary for the predictable result of providing the viewer with the convenience of 
automatically displaying the supplemental information. 

Regarding claim 10, Hidary discloses that "the supplemental data comprises a 
hyperlink to a target resource, and the initiating comprises activating the hyperlink to the 
target resource" (Col. 7 lines 20-30). 

Regarding claim 11, Hidary discloses that "the supplemental data comprises 
executable code, and the initiating comprises launching the executable code" (Col. 7 
lines 20-25, lines 54-60). 

Regarding claim 12, Hidary discloses "displaying the supplemental data 
concurrently with the primary content" (Col. 7 lines 20-30, Col. 7 lines 65-Col. 8 line 4). 

Regarding claim 13, Hidary discloses "presenting the video program within a 
hypermedia document; and controlling placement of the video program within the 
hypermedia document using the supplemental data" (Col. 7 lines 10-29, i.e. teaches of 
a JAVA enabled browser that allows a computer to retrieve web pages from a video 
program. The retrieved web pages are then synchronized with the video content). 

Regarding claim 14, claim 14 is interpreted and thus rejected for the reasons set 
forth above in the rejection of claim 8. Claim 8 describes a method of receiving 
supplemental content and initiating an enhancement action and claim 14 describes a 
computer to perform the method (Hidary: Col 5, Lines 7-8). Thus, claim 14 is rejected. 

Regarding claim 15, claim 15 is interpreted and thus rejected for the reasons set 
forth above in the rejection of claim 8. Claim 8 describes a method of receiving 
supplemental content and initiating an enhancement action and claim 15 describes 
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computer-executable instruction for performing the step of the method (Hidary: Col 7, 
Lines 11-14). Thus, claim 15 is rejected. 



Claims 21-24 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hidary in view of Brodsky, Yen, and in further view of Chen et al. (US Pat. 5,745,602), 
herein referenced as Chen. 

Regarding claim 21, Hidary discloses "a processing device (Col. 5 lines 7-20)... 
[multicasting] an enhancement action to a multicast address and a program 
enhancement listener configured to listen to the multicast address for the enhancement 
action an initiate an enhancement action based upon the supplemental data to enhance 
a video program as the video program is being played, and wherein the initiating is 
synchronized with the video based on the association of the supplemental..." (Col. 1 line 
65-Col. 2 line 12, Col. 7 lines 10-40, Col. 7 line 65-Col. 8 line 4). 

Hidary fails to explicitly disclose "a key phrase module operable by the 
processing device configured to: parse a closed captioning script to identify one or more 
key phrases in a ...script, wherein each identified key phrases has the same number of 
words, the parsing comprising determining a minimum number of words from the closed 
captioning script required to create unique phrases by iteratively selecting different word 
lengths and checking for duplication among the resulting created phrases..., wherein the 
initiating the enhancement action is performed with requiring user interaction, 
and... association of the supplemental data to one or more key phrases identified from 
the closed captioning script." 
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Brodsky discloses "a key phrase module operable by the processing device 
configured to: associate the supplemental data to one or more key phrases identified 
from the closed captioning script" (Col. 1 lines 50-62, Col. 2 lines 12-41, Col. 4 lines 4- 
17, Col. 5 lines 36-63, Col. 6 lines 12-42, i.e. Brodsky teaches that keywords are 
extracted from the closed-captioning to develop a dictionary of keywords that can be 
used to request additional information). Thus, it would have been obvious to one of 
ordinary skill in the art to apply the technique of receiving supplemental data that relates 
to one or more key phrases of a closed captioning script as taught by Brodsky, to 
improve the enhanced video programming system of Hidary for the predictable result of 
enabling the user to retrieve supplemental information pertaining to a word or group of 
words just heard (Col. 3 lines 52-60). 

The combination of Hidary and Brodsky fail to explicitly disclose that "the 
initiating the enhancement action is performed with requiring user interaction". 

Yen discloses that "the initiating the enhancement action is performed without 
requiring user interaction" (Col. 1 1 lines 42-57, i.e. Yen teaches that the foreground 
element can immediately begin displaying the information item). Thus, it would have 
been obvious to one of ordinary skill in the art to apply the technique of initiating an 
enhancement action without user interaction as taught by Yen, to improve the enhanced 
video programming system of Hidary for the predictable result of providing the viewer 
with the convenience of automatically displaying the supplemental information. 

The combination still fails to disclose "pars[ing] a ...script to identify one or more 
key phrases in a ...script, wherein each identified key phrases has the same number of 
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words, the parsing comprising determining a minimum number of words from the 
...script required to create unique phrases by iteratively selecting different word lengths 
and checking for duplication among the resulting created phrases..." 

Chen teaches "parsfing] a ...script to identify one or more key phrases in a 
...script, wherein each identified key phrases has the same number of words (Col. 1 
lines 6-10, Col. 3 lines 3-25, i.e. Chen discloses a system to automatically identify key 
phrases from a machine readable document. Initially, processor (11) selects phrases of 
two words to generate a list of phrases. Thus, each identified key phrases has the same 
number of words), the parsing comprising determining a minimum number of words 
from the ... script required to create unique phrases by iteratively selecting different 
word lengths and checking for duplication among the resulting created phrases..." (Col. 
4 lines 40-54, Col. 5 lines 20-30, Col. 6 lines 31-60, i.e. Chen teaches creating unique 
key phrases by sequentially checking different phrase lengths to determine if the phrase 
is unique. If it is determined that the phrase is a subphrase/duplicate then it will be 
removed from the key phrase list). 

Thus, it would have been obvious to one of ordinary skill in the art to apply the 
technique of parsing a script to identify a key phrase by iteratively selecting different 
word lengths and checking for duplication as taught by Chen, to improve the enhanced 
video programming system of Hidary for the predictable result of detecting key phrases 
dependent upon neither natural language processing nor corpus-dependent information 
(Col. 1 lines 39-45). 
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Regarding claim 22, Hidary discloses "that the program enhancement listener 
comprises a control embedded in a container" (Col. 7 lines 10-30, i.e. Merriam- 
Webster's Dictionary defines the word "embed" as to make something an integral part 
of. Therefore, the Java enabled browser 98 is an essential part of the client software 
106, so that the receiving terminal can access the supplemental content on the web, 
which reads on claimed "control embedded in a container"). 

Regarding claim 23, Hidary discloses that "the program enhancement listener 
comprises a control embedded in an HTML page" (Col. 7 lines 10-30, i.e. Merriam- 
Webster's Dictionary defines the word "embed" as to make something an integral part 
of. Therefore, the Java enabled browser 98 is an essential part of the client software 
106, so that the receiving terminal can access the supplemental content on the web). 

Regarding claim 24, Hidary discloses that "the program enhancement listener 
comprises a control embedded in an application" (Col. 7 lines 10-30, i.e. Merriam- 
Webster's Dictionary defines the word "embed" as to make something an integral part 
of. Therefore, the Java enabled browser 98 is an essential part of the client software 
106, so that the receiving terminal can access the supplemental content on the web). 

Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
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TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Alexander Q. Huerta whose telephone number is (571) 
270-3582. The examiner can normally be reached on M-F(Alternate Fridays Off) 7:30- 
5:00 EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Scott Beliveau can be reached on (571) 272-7343. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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/Scott Beliveau/ 

Supervisory Patent Examiner, Art Unit 2427 
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